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1 PHGB020155 
DESCRIPTION 

IMAGE RECOGNITION 

5 The present Invention relates to Image recognition systems, and In 

particular to systems adapted to provide matching of an electronically captured 
facial image with Images in a database. 

A" number of facial recognition systems are known in the art. For 
10 example, US 5,991 ,429 describes a facial recognition system in which images 
. of Individuals given security clearance access are stored on a database. 
Surveillance cameras are able to compare captured Images with those In the 
database to verify" security clearance of a target person. 

US 6,038,333 describes a recognition system suitable for Integration 
15 into a portable device such as a PDA, having a camera and a display screen. 
Face feature information from a captured Image Is compared with an Image 
database to find similar faces. Personal Information relating to the retrieved 
. faces can be displayed in order that a user can recollect previous personal 
contacts made. 

20 US 6,142,876 describes a facial recognition system for tracking players 

using gaming systems. US 6,035,055 describes a content analyser for 
determining content data of scanned Images used for Image comparison 
without retrieval of pixel data of the stored Images. US 6,188,777 describes a 
system for identifying and tracking a person's Image within a moving scene. 

25 Generally, Interpersonal communication Is made possible by the 

availability of phone numbers, e-mail addresses and other contact Information, 
which are publicised in some way or exchanged between individuals. 
Conventionally, in a social or business meeting environment, .. contact . 
information exchange has been verbal, by business card or by other written 

30 medium. More recently, personal electronic devices such as PDAs are able to 
exchange such information using infra-red or other wireless links. 
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In some circumstances, it may not be possible to achieve the proximity 
_ _.tQ.a_pfirso.D jsquir^-.ta.cequest-an -exGhange-of-^ontaet-deta^^^ -For-example-,- 
the target person may be occupied in discussion or in some other task. In 
other circumstances, it may be undesirable to initiate face-to-face contact with 
a target person until such time as the identity of, or other Information about, 
the target person is known. 

It is an object of the present invention to provide a means for obtaining 
information relating to a person, such as contact details, without face-to-face, 
verbal or other close contact. 

According to one aspect, the present invention provides an apparatus 
for obtaining personal information related to a target person, comprising: 

an Image acquisition device for capturing an image of a target person; 

a database of stored image data items each relating to one of a plurality 
of candidate persons, each image data item being associated with stored 
personal data relating to the respective candidate person; 

a search engine for matching the captured image of the target person to 
a candidate person image data item and retrieving the personal data relating 
thereto; 

an output device for presenting, to a user, the personal data relating to 
the target person; and 

control means, operable by each candidate person, to control third party 
access to the stored personal data relating to the candidate person. 

According to another aspect, the present invention provides a portable 
device for obtaining personal information related to a target person, 
comprising: 

an image acquisition device for capturing an image of a target person-; 

means for accessing a remote database of stored image data items 
each of relating to one of a plurality of candidate persons, each image data 
item being associated with personal data relating to the respective candidate 
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^ means for retrieving the personal data relating to a candidate person for 
which the captured image data of the target person matches the stored Image 
data item of the candidate person;' 

an output device for presenting, to. a user, the retrieved personal data 
5 relating to the target person; and 

control means to control third party access to the database of pefsonal 
data relating to a candidate person. 

According to another aspect, the present invention provides a system 
for providing personal information related to a target person, comprising: 
10 a database of stored image data items each relating to one of a plurality 

of candidate persons, each image data item being associated with personal 
data relating to the respective candidate person; 

means for receiving, from a remote image acquisition device, a 
captured image of a target person; 
15 a search engine for matching the captured image of the target person to 

a candidate person image data item and retrieving the personal data relating 
thereto; 

means for transmitting, to a remote output device, the personal data 
relating to the target person; and 
20 control means, operable by each candidate person, to control third party 

access to the stored personal data relating to the candidate person. 

According to another aspect, the present invention provides a method 
of obtaining information related to a target person, comprising the steps of: 
capturing an image of a target person; 
25 supplying image data from the captured image to a database of stored 

iniage data items each relating to one of a plurality of candidate persons, each 
image data item being associated with personal data relating to the respective 
candidate person; 

searching the database to match the captured image of the target 
30 person with a candidate person image data item and retrieving the personal 
data relating thereto; 

outputting the personal data relating to the target person; and 
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maintaining the database by enabling control, by each candidate 
.J...-. . ,p.^m^.QOJiiiSl..p3rly^^ to_.the .pecs^D^claia.reJ^^^ that can didate 

person. 

These and other aspects of the present invention appear in the 
5 appended claims which are incorporated herein by reference and to which the 
reader is now referred. 

Embodiments of the present invention will now be described by way of 
example only and with reference to the accompanying drawings in which: 
10 Figure 1 shows a schematic block diagram of apparatus for retrieving 

personal data corresponding to a recognised image, in which a database of 
image data is distributed across a plurality of participating devices; 

Figure 2 shows a schematic block diagram of apparatus for retrieving 
personal data corresponding to a recognised image, in which a database of 
15 image data is centralised for access by a plurality of participating devices; and 
Figure 3 shows a schematic block diagram of apparatus for retrieving 
personal data corresponding to a recognised image, in which a database of 
image data is centralised for remote access by a plurality of participating 
devices. 

20 

The present invention provides a means for one person (a "participating 
user") to obtain contact details or other personal information relating to another 
person (a 'larget person") using a process of image capture and image 
matching. Each user of the system provides their image and personal data for 
25 storage in a database of images and corresponding personal data (of 
"candidate persons"), which database is made accessible to participating 
users. 

The expressions "image" or "image data" are used herein to refer to 
data that is necessary to readily identify and/or distinguish one candidate 
30 person from other candidate persons, using an image captured by an 
appropriate image acquisition device such as a digital camera. 
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The image data may be stored in a candidate person image database in 
any appropriate form, to include compressed and uncompressed file formats, 
raw image data or pre-processed image data, or as essential parametric data 
derived from raw image data which parametric data is sufficient to facilitate 
5 image recognition and matching operations or to assist therein. 

With reference to figure 1, a first arrangement of apparatus 10, using a 
distributed database of image and personal data, is shown. 

A first participating user device 1 1 includes a digital image acquisition 
device 12, such as a digital camera, for capturing an image of a target person. 
10 The user device 11 further includes a communication device 13 for effecting 
data transfer with other user devices. The communication device may be of 
any suitable type for the intended use. 

For short range use, the communication device 13 may be an infrared 
or optical transmitter / receiver, or a short range radio device such as that 
15 prescribed by the Bluetooth standards. Alternatively, the comrnunication 
device 13 could be implemented using cellular telephone technology, such as 
GSM or GPRS. 

The user device 11 incorporates a display 14 for displaying at least 
alphanumeric data, and preferably also graphical data. The user device 

20 further includes a memory for storing image data 15 providing an image of the 
user and personal data 1 6 relating to the user. The image data 1 5 may take 
any of the forms as defined above. 

The personal data 1 6- may include any data that is specific to the user of 
the device, to include any of such items as name, address, telephone number, 

25 fax number, e-mail address, professional information including job title, 
employer details, membership of professional bodies and/or specialist interest 
groups, social information such as hobbies, membership of or affiliation with 
clubs and societies and the like. In a preferred embodiment, the personal data 
may comprise a standardised format such as that used for V-card business 

30 card exchange. 

An aspect of the invention is that the user has control over the 
accessibility to third parties of the user's personal data 16 stored on the user's 
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device 11. The user of the device 11 is able to insert, edit and restrict the 



-JiyailabiiLtyjQf tMwiii>-0^ 

For example, the control function 17 may be used to allow full access to 

the personal data by third parties, or alternatively to allow only restricted 
5 access to portions of the data at any given time. In this way, the personal data 

may be divided Into business and pleasure categories, with the user allowing 

only third party access to business data during, for example, use in a business 

environment. 



necessary data processing operations. In preferred embodiments, the user 
device 11 may be Integrated with a personal digital assistant (PDA) type 
device, a palmtop or laptop computer, a mobile telephone, a personal 

15 communicator or other suitable electronic device. 

Other participating users of the system each have a respective user 
device, illustrated as user devices 21 and 31. Each user device 21, 31 Is 
preferably substantially identical to the first user device 1 1 except, of course, in 
that there is different respective image and personal data 25, 26, 35, 36 stored 

20 therein. 

In use, a participating user (at, for example, a business conference) 
determines that he would like to obtain the contact details of (or even just 
verify the identity of) a target person in the room. The user points the image 
acquisition device 12 at the target person and captures an image of the target 

25 person. The microprocessor 1 8 performs any necessary pre-processing of the 
image, such as framing the necessary facial features and discarding other 
portions of the captured image that are unnecessary to an Image matching 
process. The pre-processing may also include data compression or 
detemrilnation of essential parametric data from the captured Image that will be 

30 used In an Image comparison operation. 



The control function may also be used to restrict access only to certain 
10 categories of third party. 

The user device also includes a microprocessdr 18 for effecting all 
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Preferably, the pre-processing operation reduces tlie quantity of image 
data necessary for comparison of tlie image data with a database of candidate 
person image data items to a bare minimum. This Is particularly relevant 
where only low bandwidth communications channels 40, 41 between devices 
are available. 

After any pre-processing, the captured image data is transmitted to 
other user devices 21, 31 using the available communications channels 40, 
41 . Preferably the captured image data is broadcast to all user devices 21 , 31 
in range of the participating user device 11. 

In a more sophisticated arrangement, a multi-cast transmission to 
selected categories of other user devices may be used. For example, the 
multi-cast address may effectively eliminate transmission to devices already 
known to the initiating user device (eg. those belonging to members of the 
same organisation, who are clearly already known to the user). In this way, 
data transmission overhead may be reduced. 

Broadly speaking, the group of other user devices 21, 31 to which the 
target Image data is broadcast or multi-cast effectively defines a database of 
image data and personal data for each of a plurality of candidate persons. 
The database is, of course, effectively a distributed database across all the 
other user devices that are within broadcast range of the user device 11. In a 
general sense, the database could be even further distributed, in that each 
user device may either hold the relevant image data and / or personal data, or 
may merely hold a data reference or pointer to the relevant data at another 
location, for example, an Intemet web page address. 

All user, devices 21, 31 receiving the target Image transmission then 
compare the target image data received with their own "candidate" image data 
25, 35. If a match is detected, the target user device that detects the match 
(eg. device 21) then detemiines whether to transmit the personal data 26 to 
the user device 1 1 originating the target image data. This will depend upon 
the settings applied by the user of device 21, using control function 27. This 
may also depend upon the identity of the user device 1 1 . 
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If the control function 27 determines that the personal data 26 may be 

transmi t ted to the user device 1 1, then the personal data is transmit t ed using 

the communlcatioris channel 40. Upon receipt of the personal data, the user 
device 1 1 displays this on display device 14 and/or saves the information to a 
5 user address book or other suitable memory location. The user device may 
alternatively or additionally present the personal data to a user as an audio 
output, for example, using a voice synthesiser. 

Preferably, the display device 14 displays at least alphanumeric 
personal data. In a further preferred embodiment, the target user device 21 

10 may transmit image data 25 back to the requesting user device 11 which 
image is displayed on the display 14. This arrangement allows the user of 
device 1 1 to verify the returned image against the real life person observed to 
ensure that the image comparison operation has been correctly executed. In 
this arrangement, a high resolution, preferably colour display 14 is needed. 

15 This function may be especially useful where the image matching 

function detects more than one possible match between the target person 
. image data and the candidate person image data items. Where multiple 
matches occur, each candidate person image (eg. 25, 35) may be displayed 
on device 1 1 for verification by the user tg decide which is the correct one. 

20 . It will be understood that in the arrangement of figure 1 , the user device 

11 effectively provides an image acquisition device 12 for capturing an image 
of the target person, a display device 14 for displaying personal data relating 
to a target person, and control means 17 for controlling access by third parties 
to the personal data 16 of the participating user stored thereon. A database of 

25 stored image data items 25, 35 relating to a plurality of candidate persons and 
a search engine for matching the captured image to a candidate person image 
data item is effectively distributed across all of the other user devices 21 , 31 . 

It will be understood that although three user devices 11, 21, 31 are 
shown, the arrangement is restricted in the number of participating devices 

30 only by practical considerations of processing and communications bandwidth. 

With reference now to figure 2, an alternative arrangement 110 is now 
described in which a centralised database 105 of candidate person image data 
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items and related personal data is provided. In figure 2, reference numerals of 
parts corresponding to those of figure 1 are numbered accordingly by the 
addition of one hundred, and need not be separately described. . 

.In the arrangement of figure 2, a first participating user device 111 
includes a digital image acquisition device 112, such as a digital camera, for 
capturing an image of a target person. The user device 1 1 1 further includes a 
communication device 113 for effecting data transfer with a central database 
105. The communication device may be of any suitable type for the intended 
use as discussed previously. 

However, in a preferred arrangement, longer range communication 
channels are likely and the use of a cellular telephone communication channel 
is preferred. 

The user device 111 also Incorporates a display 114 like in the 
arrangement of figure 1. A significant difference between the figure 2 
an-angemenl and that of figure 1 Is that image data 1 15 providing an Image of 
the user, and personal data 116 relating to the user, are not stored on the user 
device 1 1 1 , but on the database 1 05, for all users. 

The user device 11 includes control function 117 which is used to 
upload the user's Image data and personal data to the database 105, and to 
determine third party access rights thereto. The user of the device 1 1 1 is 
therefore able to insert, edit and restrict the availability of the personal data 
116 using the control function 1 17 in similar manner to that described earlier. 

The user device also Includes a microprocessor 118 for effecting all 
necessary data processing operations, as discussed above. 

Other participating users of the system each have a respective user 
device, illustrated as user devices 121 and 131. Each user device 121, 131 is 
preferably substantially identical to the first user device 111. 

In use, the operation of the arrangement of figure 2 is similar to that of 
figure 1. The significant difference is that the captured image data of the 
target is transmitted to the database 105 rather than to other user devices 121, 
131. The search is performed by apparatus in communication with the 
database, which apparatus may be associated with the database. 
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If a match (e.g. corresponding to the user of device 121) is detected in 

the da tab ase 105. th e database determines whether to transmit the personal 

data 126 to the user device 111 originating the target image data. This will 
depend upon the third party access settings applied by the user device 121 
5 using control function 127 and possibly also depends on the identity of the 
user device 111. These access settings may be stored in database 105 but 
alternatively the database (or the apparatus performing the search) may 
initiate an enquiry via communications channel 141 to device 121 to determine 
whether transmission of the personal data 126 to the first user device 111 is 
10 permitted. 

If third party access rights permit, the personal data 126 is transmitted 
to the user device 111 using the communications channel 140. Upon receipt 
of the fDersonal data, the user device 111 displays this on display 114 and/or 
saves the information to a user address book or other suitable memory 
15 location. 

It will be understood that in the arrangement of figure 1 , the user device 
1 1 effectively provides an image acquisition device for capturing an image of 
the target person, a display device for displaying personal data relating to a 
target person, and control means for controlling access by third parties to the 

20 personal data of the participating user stored thereon. A database of stored 
image data items relating to a plurality of candidate persons and a search 
engine for matching the captured image to a candidate person image data 
item is effectively provided centrally to all of the other user devices. 

With the centralised database system of figure 2, there is not 

25 necessarily any enforced proximity of user devices making queries (as there 
could be by the communication channel in figure 1). Therefore, it may be 
desirable to impose some geographical limitations on the extent of search. 
The database 1 05 may limit the search of candidate image data items to those 
that relate to user devices clearly in the same geographical area as the 

30 participating user device that sends the target image data. 

This geographical search limitation can be achieved in a number of 

ways. Where a cel^ telephone communication channel js ^e[ng used, the 
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geographical location may be obtained from the operating cells of the relevant 
devices. Alternatively, each user device may routinely provide a location 
update to the database according to a GPS fix.- Alternatively, a simple user 
registration procedure may be provided, in which each user attending a 
conference, for example, may voluntarily enter their location or attendance at 
the conference. Alternatively, a conference organiser may provide a local 
database specific to the conference that can be pre-loaded with the relevant 
image data and / or associated personal data of conference attendees upon 
registration of the participants. 

With reference to figure 3, a modification of the arrangement of figure 2 
is illustrated. In this arrangement 210, a centralised database 205 is still 
accessed by user devices 211, 221, 231. Communication with the database 
205 is effected using cellular telephone links 240, 241, 242, which connect the 
user devices vyith a network 250, Internet gateway 251 and internet 252, to a 
server 253 coupled to the database 205. 

In this arrangement, the server 253 provides the search engine for 
matching the captured Image data of the target person to a candidate person 
image data item in the database 205. 

it will be understood that a large number of image recognition and 
matching systems are available in the present state of the art, many of which 
will be suitable for implementation of the systems described herein. 

Preferably, the systems described herein can be implemented on 
existing available hardware, such as PDA type devices or mobile 
communicators that have the requisite cameras, by providing suitable software 
for download, possibly over the internet. 

Other embodiments are intentionally within the scope of the 
accompanying claims. 
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CLAIMS 

1. Apparatus for obtaining personal information related to a target 
person, comprising: 

5 an image acquisition device for capturing an image of a target person; 

a database of stored image data items each relating to one of a plurality 
of candidate persons, eacli image data item being associated with stored 
personal data relating to the respective candidate person; 

a search engine for matching the captured image of the target person to 
10 a candidate person image data item and retrieving the personal data relating 
. thereto; 

an output device' for presenting, to a user, the personal data relating to 
. the target person; and 

control means, operable by each candidate person, to control third party 
15 access to the stored personal data relating to the candidate person. 

2. The apparatus of claim 1 in which the database is a distributed 
database, candidate persons each having a portable device for storing their 
own image data items and personal data which may be accessed by the 

20 search engine using a wireless communication channel. 

3. The apparatus of claim 2 in which the control means comprises an 
access control function provided on each portable device. 

25 4. The apparatus of claim 1 in which the database includes a central 

repository accessible to a plurality of remote portable devices using a wireless 
communication channel. 

5. The apparatus of claim 4 in which the control means is a distributed 
30 . control means, candidate persons each having a device for storing their own 
image data items and personal data onto the database and determining third 
party access rights thereto. 
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6. The apparatus of claim 1 in wliicli the image acquisition device, 
output device and control means are integrated into a portable electronic 
device. 

7. The apparatus of claim 6 in which the portable electronic device is 
any of a personal digital assistant, personal computer or mobile telephony 
device. 

8. The apparatus of claim 6 or claim 7 in which the portable electronic 
device further includes communication means for communication with a 
remotely located database and the search engine, 

9. The apparatus of claim 1 in which the output device is a display 
device for displaying the personal data relating to the target person. 

10. A portable device for obtaining personal information related to a 
target person, comprising: 

an image acquisition device for capturing an image of a target person; 

means for accessing a remote database of stored image data items 
each of relating to one of a plurality of candidate persons, each image data 
item being associated with personal data relating to the respective candidate 
person; 

means for retrieving the personal data relating to a candidate person for 
which the captured image data of the target person matches the stored image 
data item of the candidate person; 

an output device for presenting, to a user, the retrieved personal data 
relating to the target person; and 

control means to control third party access to the database of personal 
data relating to a candidate person. 
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11. The portable device of claim 10 in which the means for accessing 
and the means fo r retriev ing in clude a wireles s co mmunication device. 

12. The portable device of claim 10 integrated with any of a personal 
5 digital assistant, personal computer or mobile telephony device. 

13. The portable device of claim 11 in which the wireless 
communication device is adapted to communicate with a plurality of 
corresponding devices, the corresponding devices together forming the remote 

10 database. 

14. The portable device of claim 10 in which the output device is a 
display device for displaying the personal data relating to the target person. 

15 15. A system for providing personal Information related to a target 

person, comprising: 

a database of stored irfiage data items each relating to one of a plurality 

of candidate persons, each image data item being associated with personal 

data relating to the respective candidate person; 
20 means for receiving, from a remote image acquisition device, a 

captured image of a target person; 

a search engine for matching the captured image of the target person to 

a candidate person image data item and retrieving the personal data relating 

thereto; 

25 means for transmitting, to a remote output device, the personal data 

relating to the target person; and 

control means, operable by each candidate person, to control third party 
access to the stored personal data relating to the candidate person. 



30 16. The system of claim 15 in which the means for receiving and the 

means for transmitting include a wireless communication link. 
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17. The system of claim 15 or claim 16 in which the means for 
receiving and the means for transmitting include an internet connection. 

18. A method of obtaining information related to a target person, 
comprising the steps of: 

. capturing an image of a target person; 
supplying image data from the captured image to a database of stored 
image data items each relating to one of a plurality of candidate persons, each 
image data item being associated with personal data relating to the respective 
candidate person; 

searching the database to match the captured image of the target 

person with a candidate person Image data Item and retrieving the personal 

data relating thereto; 

outputting the personal data relating to the target person; and 
maintaining the database by enabling control, by each candidate 

person, of third party access to the personal data relating to that candidate 

person. 

19. A computer program product, comprising a computer readable 
medium having thereon computer program code means adapted, when said 
program is loaded onto a computing apparatus, to make the computing 
apparatus form the device of any one of claims 1 0 to 1 4. 

20. A computer program, distributable by. electronic data transmission, 
comprising computer program code means adapted, when said program is 
loaded onto a computing apparatus, to make the computing apparatus form 
the device of any one of claims 1 0 to 14. 
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ABSTRACT 

IMAGE RECOGNITION 

5 A system and method for obtaining information relating to a person, 

sucfi as contact details, witliout face-to-face, verbal or otiier contact. The 
apparatus uses an image recognition system for obtaining personal 
information related to a target person, and Includes: an image acquisition 
device for capturing an image of a target person; a database of stored image 

10 data items each relating to one of a plurality of candidate persons, each image 
data item being associated with stored personal data relating to the respective 
candidate person; a search engine for matching the captured image of the 
target person to a candidate person image data item and retrieving the 
personal data relating thereto; a display device for displaying the personal data 

15 relating to the target person; and control means, operable by each candidate 
person, to control third party access to the stored personal data relating to the 
candidate person. 
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